1. Field of the Invention
The present invention relates to an image processing method, an image processing device and a recording medium, and in particular, to an image processing method which carries out spatial frequency enhancement processing on an original image signal (image data) obtained by a digital still camera under preferred processing conditions, an image processing device to which the image processing method can be applied, and a recording medium on which a program for a computer to execute the image processing method is recorded.
2. Description of the Related Art
Conventionally, with the object of turning the image quality of an image, which is expressed by an original image signal obtained by image pick-up using a video camera or photography using a digital still camera or the like, into an image quality which is visually preferable, spatial frequency enhancement processing for enhancing the sharpness of the image expressed by the above original image signal, which is known as sharpness enhancement processing, is carried out.
In carrying out sharpness enhancement processing such as that described above, an image of an image quality that is visually more preferable can be obtained by setting processing conditions according to a combination of an input device and an output device. The input device is used in obtaining an original image signal and the output device is used in outputting the image based on the original image signal. Thus, examinations have conventionally been made of preferred processing conditions for sharpness enhancement processing in accordance with a combination of various types of input devices and output devices.
Japanese Patent Application Laid-Open (JP-A) No. 61-109550 has made an examination of processing conditions of the sharpness enhancement processing in which a digital X-ray image is inputted, and the processed image is outputted onto a CRT monitor or a X-ray film.
In this way, the optimization of the processing conditions of sharpness enhancement processing in accordance with the combination of the input device and the output device has become an essential technique in realizing a high image quality.
However, since the digital still camera is a device that has been put to practical use relatively recently and only a limited number of manufacturers deal in both devices, i.e., the digital still camera and a printer for recording an original image signal, which is obtained by photography using the digital still camera, on a recording medium, there has been a problem in that, the conditions obtained as processing conditions for sharpness enhancement processing at the time when an image is recorded on a recording medium based on an original image signal obtained by photography using a digital still camera have not been necessarily the optimum ones.
Further, some types of recent digital still cameras carry out sharpness processing within the camera. In these types of digital still cameras, an output thereof is adjusted to the sharpness that is preferable for viewing on a monitor. Moreover, the image setting, which is preferable for viewing on a monitor, differs from manufacturer to manufacturer among manufacturers of digital still cameras. As a result, there exist a variety of sharpness images, and even when an image, which is preferable for viewing on a monitor, is outputted as it is by a printer, the sharpness of the image may be insufficient.
The present invention has been attained to solve the above-described problems, and an object thereof is to provide an image processing method, an image processing device and a recording medium in which spatial frequency enhancement processing can be carried out under preferred processing conditions on original image signals obtained by a digital camera, which include an original image signal of an image on which sharpness processing is not effected by a digital still camera or an original image signal of an image on which sharpness processing has been effected by a digital still camera so as to look preferable for viewing on a monitor.
In order to achieve the above object, an image processing method relating to a first aspect of the present invention carries out spatial frequency enhancement processing on an original image signal obtained by photography using a digital still camera in such a manner that the maximum value of responses of the spatial frequency characteristics of an image signal on which the spatial frequency enhancement processing has been effected is a value in the range of 1.5 to 3.0 times the maximum value of responses of the spatial frequency characteristics of the original image signal on which the spatial frequency enhancement processing has not been effected yet.
The inventors carried out a subjective evaluation experiment that utilized an image which was the result of spatial frequency enhancement processing on an original image signal that had been obtained by photography using a plurality of existing digital still cameras.
The respective conditions of the subjective evaluation experiment were as follows:
evaluators: 8 persons;
types of digital still cameras that were used for the evaluation: five types, i.e., digital still cameras from A to E;
degree of spatial frequency enhancement processing: the maximum value of responses of the spatial frequency characteristics in the image signal on which the spatial frequency enhancement processing has been effected is 1.5 times, 2.0 times, 2.5 times, 3.0 times, 3.5 times, 4.0 times or 4.5 times the maximum value of responses of the spatial frequency characteristics in the original image signal;
number of scenes to be evaluated: 10 scenes for each type of digital camera (5 scenes each having an image mainly of a person, and 5 scenes each having an image mainly of a landscape);
evaluation method: each image was compared with the original image so as to be given one of the following 5 grades
⊚: improved considerably
∘: improved
xcex94: in comparison with the original image, some parts were improved but others were deteriorated, or equivalent to the original image
X: deteriorated
XX: deteriorated considerably.
In table 1, an example of evaluation results for 5 scenes each having an image mainly of a landscape is shown. In table 2, an example of evaluation results for 5 scenes each having an image mainly of a person is shown.
Table 3 shows the results of the evaluation of all the scenes to be evaluated by all the evaluators. These results have been made numerical by the expression, the number of scenes to be evaluatedxc3x97 the number of persons. (For example, if chosen by all the evaluators, the result is 10 scenesxc3x975 typesxc3x978 persons=400.)
From the results of the subjective evaluation described above, the information was obtained: when the maximum value of the spatial frequency characteristics in the image signal on which the spatial frequency enhancement processing has been effected is in the range of 1.5 to 3.0 times the maximum value of responses of the spatial frequency characteristics of the original image signal, a larger proportion of evaluators (equal to or more than 50%) answered that the image was improved in comparison with the original image, and thus images of visually preferable image quality can be obtained in this range. Particulary, the range of 2.0 to 2.5 times is more preferable.
Based on the evaluation results described above, according to the image processing method of the first aspect, as spatial frequency enhancement processing is carried out on an original image signal obtained by photography using a digital still camera in such a manner that the maximum value of responses of the spatial frequency characteristics of an image signal on which the spatial frequency enhancement processing has been effected is in the range of 1.5 to 3.0 times the maximum value of responses of the spatial frequency characteristics of the original image signal on which the spatial frequency enhancement processing has not been effected yet, the spatial frequency enhancement processing can be carried out under preferred processing conditions on the original image signal obtained by photography using a digital still camera.
In the meantime, if conventional spatial frequency enhancement processing is carried out on an original image signal, noise in the image signal is also enhanced. Further, as image signals obtained by photography using a digital still camera are generally compressed by the JPEG method, many image signals contain a good deal of noise. Accordingly, when conventional spatial frequency enhancement processing is carried out on an image signal obtained by photography using a digital still camera, the image obtained as the result has a low image quality whose noise is conspicuous.
Thus, enhancement to portions having small contrast values of the original image signal is suppressed because these portions are considered to be noise components. Only portions having large contrast values are enhanced because these portions are considered to be edge components. As a result, a sharp image can be produced without enhancing noise.
On the other hand, in images expressed by original image signals that are obtained by photography using a digital still camera, there is a general tendency for dark images to contain more noise. For this reason, for example, when conventional spatial frequency enhancement processing is carried out on an image signal of an image photographed in an underexposed manner, noise becomes especially conspicuous. Thus, by suppressing the degree of enhancement when the density of the image is high (when the image is dark), deterioration of the image quality resulting from noise can be prevented.
In view of the above-described points, in an image processing method relating to a second aspect of the present invention, the spatial frequency enhancement processing in the first aspect is carried out based on the following expression:
Sproc=Sorg+G(Sorg)xc3x97F(Sorgxe2x88x92Sus)xe2x80x83xe2x80x83(1)
Here, Sorg expresses the original image signal, Sus expresses a non-sharp mask signal, G (Sorg) expresses a function dependent on the original image signal Sorg, and F (Sorgxe2x88x92Sus) expresses a function dependent on a contrast value of the original image signal. It should be noted that the non-sharp mask signal Sus used here refers to a signal of a non-sharp image whose original image signal is soft-focused so as to contain exclusively the components whose spatial frequencies are lower than the predetermined spatial frequency (what is known as a soft-focus image).
F (Sorgxe2x88x92Sus) is a function that depends on a contrast value of an original image signal. Therefore, by incorporating the function into the above expression (1), for example, such operations as described above, i.e., suppressing enhancement of the portions that have small contrast values of the original image signal by considering these portions to be noise components and enhancing only the portions that have large contrast values by considering these portions to be edge components, are made possible.
Further, G (Sorg) is a function that depends on an original image signal. Therefore, by incorporating the function into the above expression (1), for example, such operations as described above, i.e., suppressing the degree of enhancement when the density of the image is high (when the image is dark), are made possible.
In this way, according to the image processing method relating to the second aspect, the same effect as that of the first aspect can be produced, and as the spatial frequency enhancement processing is carried out based on the expression incorporating the function dependent on the contrast value of the original image signal as well as the function dependent on the original image signal, an image signal whose noise is not conspicuous and which has the high image quality can be obtained.
In the mean time, a grain of noise, which is contained in an original image signal obtained by photography using a digital still camera, is generally large in comparison with an ordinary photographic image. Therefore, if the low frequency band in the spatial frequency characteristics is enhanced to excess, noise becomes considerably conspicuous. For this reason, in the spatial frequency enhancement processing effected on the original image signal obtained by photography using a digital still camera, it is preferable to suppress the enhancement of the above low frequency band. Particularly, in regard to the spatial frequency characteristics of the non-sharp mask signal Sus, the inventors have obtained information that on an image outputted by using the image signal on which the spatial frequency enhancement processing has been effected by utilizing the non-sharp mask signal, when a response in the spatial frequency of 0 to 0.5 cycle/mm is made more than 1.2 times larger than a response before the spatial frequency enhancement processing, a visually preferable image cannot be obtained.
Thus, in an image processing method relating to a third aspect of the present invention, the spatial frequency characteristics of the non-sharp mask signal Sus in the second aspect is such that on an image outputted by using an image signal on which the spatial frequency enhancement processing has been effected, a response of the spatial frequency of 0 to 0.5 cycle/mm is equal to or less than 1.2 times a response before the spatial frequency enhancement processing.
In this way, according to the image processing method relating to the third aspect, as the spatial frequency characteristics of the non-sharp mask signal Sus is such that on an image output using by an image signal on which the spatial frequency enhancement processing has been effected, a response of the spatial frequency of 0 to 0.5 cycle/mm is equal to or less than 1.2 times the response before the spatial frequency enhancement processing, the degree of enhancement in the low frequency band can be suppressed. As a result, an image signal whose noise is less conspicuous and which has a high image quality can be obtained.
As described above, enhancement of the portions having small contrast values of the original image signal is suppressed because these portions are considered to be noise components. Only the portions having large contrast values are enhanced because these portions are considered to be edge components. As a result, a sharp image can be produced without enhancing noise.
Thus, in an image processing method relating to a fourth aspect of the present invention, the function F (Sorgxe2x88x92Sus) in the second or third aspect has the characteristics that, which is obtained by subtracting the non-sharp mask signal Sus from the original image signal Sorg, is smaller than a predetermined threshold value, the function F (Sorgxe2x88x92Sus) is smaller than the contrast value. In an image processing method relating to a fifth aspect of the present invention, the function F (Sorgxe2x88x92Sus) in the fourth aspect has the characteristics that, when the absolute value of the contrast value is smaller than a predetermined threshold value, the function F (Sorgxe2x88x92Sus) is 0.
In this way, in the image processing methods relating to the fourth and fifth aspects, the same effect as that of the second or third aspect can be produced, and because the characteristics of the function F (Sorgxe2x88x92Sus) are such that, when the absolute value of a contrast value, i.e. the magnitude of the contrast value, is smaller than a predetermined threshold value, the magnitude of the function F (Sorgxe2x88x92Sus) is smaller than the contrast value, an image which is sharp and whose noise is not conspicuous can be produced.
As to the predetermined threshold value in the fourth and fifth aspects, based on the subjective evaluation utilizing the original image signals obtained by photography using many types of existing digital still cameras, the inventors have obtained information that the threshold value is preferably in the range of 2 to 10% of the maximum value of the original image signal.
In the meantime, as described above, when the density of the image expressed by the original image signal obtained by photography using a digital still camera is high (when the image is dark), deterioration of the image quality resulting from noise can be prevented by suppressing the degree of enhancement.
Thus, in an image processing method relating to a sixth aspect, the function G (Sorg) in any aspect of the second to fifth aspects has the characteristics in which the value thereof decreases, as the density of an image on which the spatial frequency enhancement processing has not been effected yet becomes higher.
In this way, in the image processing method relating to the sixth aspect, the same effect as that of the second to fifth aspects can be produced, and as the characteristics of the function G (Sorg) in the second to fifth aspects are such that the value thereof decreases as the density of an image on which the spatial frequency enhancement processing has not been effected yet becomes higher, deterioration of the image quality resulting from noise can be prevented.
In the meantime, when an original image signal which is obtained by photography using a digital still camera is used for outputting onto a photosensitive material or the like, enlargement processing or reduction processing may sometimes be carried out on the above original image signal according to the size and the like of the output.
FIG. 18A shows an example of the spatial frequency characteristics of an image signal that is enlarged by 8/7 times by a linear interpolation method, and FIG. 18B shows an example of the spatial frequency characteristics of an image signal that is enlarged by 8/7 times by a cubic B-spline interpolation method. From FIGS. 18A and 18B, it is understood that the spatial frequency characteristics vary depending on the type of enlargement/reduction method.
With this point in view, an image processing method relating to a seventh aspect of the present invention carries out enlargement processing or reduction processing on an original image signal obtained by photography using a digital still camera, and, in carrying out spatial frequency enhancement processing, adjusts a response of the spatial frequency characteristics of an image signal on which the spatial frequency enhancement processing has been effected, according to the spatial frequency characteristics of the enlargement processing or the reduction processing.
In this way, according to the image processing method relating to the seventh aspect, as a response of the spatial frequency characteristics of an image signal on which the spatial frequency enhancement processing has been effected is adjusted according to the spatial frequency characteristics of enlargement processing or reduction processing, the spatial frequency enhancement processing can be carried out under preferred processing conditions corresponding to the type of enlargement method or reduction method.
Here, if a response of the spatial frequency characteristics of enlargement processing or reduction processing is added to the results of the subjective evaluation shown in FIG. 3, as in an image processing method of an eighth aspect of the present invention, it is preferable that the adjustment in the seventh aspect is such that the maximum value of responses of the spatial frequency characteristics, in which the spatial frequency characteristics of the above-described enlargement processing or the above-described reduction processing and the spatial frequency characteristics of an image signal on which the spatial frequency enhancement processing has been effected are combined, is a value in the range of 1.0 to 2.5 times the maximum value of responses of the spatial frequency characteristics of the original image signal on which the spatial frequency enhancement processing has not been effected yet.
FIG. 18C shows the spatial frequency characteristics of an image signal that is enlarged by 10/7 times by the cubic B-spline interpolation method (except for the scale of enlargement, the conditions are the same as those show in FIG. 18B). From FIGS. 18B and 18C, it is understood that even if the type of enlargement/reduction method is the same, the spatial frequency characteristics vary according to the scale of enlargement or the scale of reduction.
Further, for example, when an original image is small and thus required to be enlarged considerably in order to make a print, the image quality of an enlarged image greatly deteriorates. Therefore, if spatial frequency enhancement processing is carried out on an image signal expressing the enlarged image, the image quality further deteriorates.
From the above-described point of view, as in an image processing method relating to a ninth aspect of the present invention, it is preferable that the adjustment in the seventh aspect is an adjustment in accordance with the scale of enlargement when the enlargement processing is carried out, or with the scale of reduction when the reduction processing is carried out.
More specifically, as the scale of enlargement increases, the response of the spatial frequency characteristics of the image signal on which the spatial frequency enhancement processing has been effected is decreased, and as the scale of reduction increases, the above response is increased.
Further, an image processing device relating to a tenth aspect of the present invention includes controlling means for effecting control so that spatial frequency enhancement processing is carried out on an original image signal obtained by photography using a digital still camera in such a manner that the maximum value of responses of the spatial frequency characteristics of an image signal on which the spatial frequency enhancement processing has been effected is a value in the range of 1.5 to 3.0 times the maximum value of responses of the spatial frequency characteristics of the original image signal on which the spatial frequency enhancement processing has not been effected yet.
According to the tenth aspect of the present invention, control is effected by the controlling means so that spatial frequency enhancement processing is carried out on an original image signal obtained by photography using a digital still camera in such a manner that the maximum value of responses of the spatial frequency characteristics of an image signal on which the spatial frequency enhancement processing has been effected is a value in the range of 1.5 to 3.0 times the maximum value of responses of the spatial frequency characteristics of the original image signal on which the spatial frequency enhancement processing has not been effected yet.
In this way, according to the image processing device of the tenth aspect, as control is effected so that spatial frequency enhancement processing is carried out on an original image signal obtained by photography using a digital still camera in such a manner that the maximum value of responses of the spatial frequency characteristics of an image signal on which the spatial frequency enhancement processing has been effected is a value in the range of 1.5 to 3.0 times the maximum value of responses of the spatial frequency characteristics of the original image signal on which the spatial frequency enhancement processing has not been effected yet, as in the first aspect of the present invention, the spatial frequency enhancement processing can be carried out under preferred processing conditions on the original image signal obtained by photography using a digital still camera.
Further, an image processing device relating to an eleventh aspect of the present invention includes enlargement/reduction means for carrying out enlargement processing or reduction processing on an original image signal obtained by photography using a digital still camera, and adjusting means for adjusting a response of the spatial frequency characteristics of an image signal on which the spatial frequency enhancement processing has been effected, according to the spatial frequency characteristics of the enlargement processing or the reduction processing.
According to the eleventh aspect of the present invention, enlargement processing or reduction processing is carried out by the enlargement/reduction means on an original image signal obtained by photography using a digital still camera, and a response of the spatial frequency characteristics of an image signal on which the spatial frequency enhancement processing has been effected is adjusted by the adjusting means according to the spatial frequency characteristics of the enlargement processing or the reduction processing.
In this way, according to the image processing device of the eleventh aspect, as a response of the spatial frequency characteristics of an image signal on which spatial frequency enhancement processing is effected is adjusted according to the spatial frequency characteristics of the enlargement processing or the reduction processing, as in the seventh aspect of the present invention, the spatial frequency enhancement processing can be carried out under preferred processing conditions in accordance with the type of enlargement method or reduction method.
Further, a recording medium relating to a twelfth aspect of the present invention has a program recorded thereon for allowing a computer to execute processing that includes a step in which spatial frequency enhancement processing is carried out on an original image signal obtained by photography using a digital still camera in such a manner that the maximum value of responses of the spatial frequency characteristics of an image signal on which the spatial frequency enhancement processing is effected is a value in the range of 1.5 to 3.0 times the maximum value of responses of the spatial frequency characteristics of the original image signal on which the spatial frequency enhancement processing has not been effected yet.
Since the recording medium relating to the twelfth aspect of the present invention has a program recorded thereon for allowing a computer to execute processing that includes the above step, in other words, processing according to the image processing method relating to the invention of the first aspect, spatial frequency enhancement processing can be carried out under preferred processing conditions on an original image signal obtained by photography using a digital still camera, as in the first aspect of the present invention, when the computer reads and executes the program recorded on the recording medium.
Further, a recording medium relating to a thirteenth aspect of the present invention has a program recorded thereon for allowing a computer to execute processing that includes a first step in which enlargement processing or reduction processing is carried out on an original image signal obtained by photography using a digital still camera, and a second step in which a response of the spatial frequency characteristics of an image signal on which the spatial frequency enhancement processing has been effected is adjusted according to the spatial frequency characteristics of the enlargement processing or the reduction processing.
Since the recording medium relating to the thirteenth aspect of the present invention has a program recorded thereon for allowing a computer to execute processing that includes the above-described first and second steps, in other words, processing according to the image processing method relating to the seventh aspect of the present invention, spatial frequency enhancement processing can be carried out under preferred processing conditions in accordance with the type of enlargement method or reduction method, as in the seventh aspect of the present invention, when the computer reads and executes the program recorded on the recording medium.